Apparatus and Method for Controlling Vehicle

ABSTRACT

An apparatus for controlling a vehicle includes a sensor that detects motion and approach of an object around the vehicle, a camera that obtains a motion image and a facial image of the object around the vehicle, and a controller that generates a graphical user interface (GUI) based on the obtained motion and facial images of the object around the vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Korean Patent Application No.10-2018-0121244, filed in the Korean Intellectual Property Office onOct. 11, 2018, which application is hereby incorporated herein byreference.

TECHNICAL FIELD

The present disclosure relates to an apparatus and method forcontrolling a vehicle.

BACKGROUND

A technology for emphasizing communication between a vehicle and a userhas been developed in consideration of user convenience as well as animprovement in vehicle driving performance. For example, a technologyfor integrating a welcome system into an external lamp of a vehicle hasbeen developed.

A welcome system is a system that reacts as if a vehicle welcomes a userwhen the user closely approaches the vehicle. Examples of an operatingsystem associated with the welcome system include lighting of a lamp(e.g., a puddle lamp, a positioning lamp, a tail lamp, a pocket lamp, orthe like), movement of an outside mirror, a welcome sound operation, andthe like.

The welcome system is a new system that is capable of satisfying auser's emotional needs, as well as providing ease of recognition of avehicle to the user.

However, the welcome system fails to reflect the user's status due to auniform display and a uniform operating mode.

Meanwhile, a smart key system has been introduced for user convenience.A user carrying a smart key can control a door lock of a vehicle and canstart the vehicle. Furthermore, functions are implemented such thatcomponents in a vehicle are not directly operated and controlled by auser, but automatically controlled through sensors.

In recent years, a technology for controlling a door lock of a vehicleby recognizing a user's face has been developed. A user may approach avehicle to allow his/her face to be recognized. However, when the faceis not accurately recognized, the cause is not fed back andre-recognition is frequently requested, thereby inconveniencing theuser.

SUMMARY

Embodiments of the present disclosure can solve the above-mentionedproblems occurring in the prior art while advantages achieved by theprior art are maintained intact.

An aspect of the present disclosure provides a vehicle control apparatusand method for outputting a graphical user interface to provide feedbackthat a face is not detected or outputting a graphical user interfaceaccording to a user's emotional state.

The technical problems to be solved by the present inventive concept arenot limited to the aforementioned problems, and any other technicalproblems not mentioned herein will be clearly understood from thefollowing description by those skilled in the art to which the presentdisclosure pertains.

According to an aspect of the present disclosure, an apparatus forcontrolling a vehicle includes a sensor that detects approach of anobject around the vehicle, a camera that obtains a motion image and afacial image of the object around the vehicle, and a controller thatgenerates a graphical user interface (GUI), based on the obtained motionand facial images of the object around the vehicle.

The controller may extract a first feature vector, based on the motionimage of the object around the vehicle.

The first feature vector may include at least one of a specific gestureand a body shape of the object around the vehicle.

The controller may extract a second feature vector from the facial imageand may determine whether the object is a driver, based on the secondfeature vector.

The second feature vector may include at least one of a facialexpression and a complexion.

The controller may generate multi-modal data, based on the secondfeature vector and the first feature vector matching the second featurevector.

The controller may determine an emotional state of the driver, based onthe multi-modal data.

The controller may differently generate the graphical user interface(GUI) according to the emotional state.

The controller may control locking/unlocking of doors of the vehiclewhen determining that the object around the vehicle is the driver.

The controller may determine whether a face of the object around thevehicle is detected, based on the motion image, and when determiningthat the face is not detected, the controller may generate a graphicaluser interface to provide feedback on a reason why the face is notdetected.

The apparatus may further include a window display on which thegraphical user interface (GUI) is output.

According to another aspect of the present disclosure, a method forcontrolling a vehicle includes a step of obtaining a motion image and afacial image of an object around the vehicle, a step of determining adriver's emotional state, based on the obtained motion and facial imagesof the object around the vehicle, and a step of outputting a graphicaluser interface generated based on the driver's emotional state.

The method may further include a step of determining whether a face ofthe object around the vehicle is detected, based on the motion image,prior to the step of obtaining the facial image of the object around thevehicle, and a step of outputting a graphical user interface to providefeedback on a reason why the face is not detected, when determining thatthe face of the object around the vehicle is not detected.

The method may further include a step of extracting a first featurevector from the motion image of the object around the vehicle after thestep of obtaining the motion image of the object around the vehicle.

The first feature vector may include at least one of a specific gestureand a body shape of the object around the vehicle.

The method may further include a step of extracting a second featurevector from the facial image of the object around the vehicle anddetermining whether the object is the driver, based on the secondfeature vector, after the step of obtaining the facial image of theobject around the vehicle.

The second feature vector may include at least one of a facialexpression and a complexion.

The step of determining the driver's emotional state based on theobtained motion and facial images of the object around the vehicle mayinclude a step of generating multi-modal data, based on the secondfeature vector and the first feature vector matching the second featurevector and a step of determining the driver's emotional state, based onthe multi-modal data.

The method may further include a step of controlling locking/unlockingof doors of the vehicle when determining that the object around thevehicle is the driver.

In the step of outputting the graphical user interface generated basedon the driver's emotional state, the graphical user interface may bedifferently output according to the driver's emotional state.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentdisclosure will be more apparent from the following detailed descriptiontaken in conjunction with the accompanying drawings:

FIG. 1 is a block diagram illustrating an apparatus for controlling avehicle according to an embodiment of the present disclosure;

FIG. 2 is a flowchart illustrating a method for controlling a vehicleaccording to an embodiment of the present disclosure;

FIG. 3 is a flowchart illustrating a method for controlling a vehicleaccording to another embodiment of the present disclosure;

FIG. 4 is a view illustrating a camera included in the vehicle controlapparatus according to an embodiment of the present disclosure;

FIG. 5 is a view illustrating the angle of view of the camera accordingto an embodiment of the present disclosure;

FIG. 6 is a view illustrating a display type according to an embodimentof the present disclosure;

FIGS. 7 to 13 are views illustrating graphical user interfaces displayedaccording to embodiments of the present disclosure; and

FIG. 14 is a block diagram illustrating a configuration of a computingsystem that executes a method according to an embodiment of the presentdisclosure.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Hereinafter, exemplary embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings. Itshould be understood that even if shown in different drawings, identicalcomponents are provided with identical reference numerals in thedrawings. Furthermore, in describing the embodiments of the presentdisclosure, detailed descriptions related to well-known functions orconfigurations will be omitted when they may make subject matters of thepresent disclosure unnecessarily obscure.

Terms, such as “first”, “second”, “A”, “B”, “(a)”, “(b)”, and the like,may be used herein to describe components of the present disclosure.Such terms are only used to distinguish one component from anothercomponent, and the substance, sequence, order, or number of thesecomponents is not limited by these terms. Unless otherwise defined, allterms used herein, including technical and scientific terms, have thesame meaning as those generally understood by those skilled in the artto which the present disclosure pertains. Such terms as those defined ina generally used dictionary are to be interpreted as having meaningsequal to the contextual meanings in the relevant field of art, and arenot to be interpreted as having ideal or excessively formal meaningsunless clearly defined as having such in the present application.

FIG. 1 is a block diagram illustrating an apparatus for controlling avehicle according to an embodiment of the present disclosure.

As illustrated in FIG. 1, the vehicle control apparatus according to theembodiment of the present disclosure may include a sensor 10, a camera20, a window display 30, and a controller 40.

The sensor 10 may detect motion and approach of an object around thevehicle. To this end, the sensor 10 may be implemented with a proximitysensor and may be located on a lateral side of the vehicle. Here, themotion of the object around the vehicle may refer to motions of allobjects that move around the vehicle, and the object may include a user.

The proximity sensor may detect whether any object approaches a presetcontrol zone, without mechanical contact. The proximity sensor mayinclude an IR sensor, a capacitive sensor, an RF sensor, or the like.

According to an embodiment, the sensor 10 may detect motion and approachof a surrounding object within a predetermined distance from thevehicle. Here, the predetermined distance may be defined to be 30 cm.

The camera 20 may obtain a motion image and a facial image of the objectaround the vehicle. To this end, according to an embodiment, the camera20 may be included in a pillar (that is, a center pillar (a B pillar))of the vehicle.

In the case where the camera 20 is included in the side pillar, thecamera 20 has a vertical angle of view ranging from 42° to 70° and ahorizontal angle of view ranging from 45° to 55°.

The window display 30 may include a door glass onto which an image isprojected in a projector manner.

According to an embodiment, the window display 30 may be implemented insuch a manner that images are projected onto door glasses on oppositesides of the front seats of the vehicle.

The controller 40 may control an overall operation of the vehiclecontrol apparatus of the present disclosure. When determining that themotion of the object around the vehicle is detected, the controller 40may extract a first feature vector from the motion image of the objectobtained from the camera 20.

Here, the first feature vector is intended to detect the feature of themotion of the object around the vehicle. According to an embodiment, thefirst feature vector may include a specific gesture or a body shape ofthe object around the vehicle. A face may be included in the body shape.Since the detected face does not have accuracy sufficient to determinewhether the object is a driver or not, it is not easy to determinewhether the object is the driver or not, based on only the faceextracted as the first feature vector.

Therefore, the controller 40 may extract the specific gesture or thebody shape of the object around the vehicle from the motion image of theobject and may detect the feature of the motion of the object around thevehicle from the specific gesture or the body shape.

When determining that the object around the vehicle approaches thevehicle, the controller 40 may determine whether a face of the objectaround the vehicle is detected. When determining that the face of theobject around the vehicle is detected, the controller 40 may extract asecond feature vector from the facial image of the object obtained fromthe camera 20.

Here, the second feature vector is intended to detect a facialexpression. According to an embodiment, the second feature vector mayinclude facial features (eyes, nose, and mouth) and a complexion.

Accordingly, the controller 40 may extract the facial features or thecomplexion from the facial image of the object around the vehicle andmay detect the feature of the facial expression from the facial featuresor the complexion.

The controller 40 may determine who the object around the vehicle is,based on the extracted second feature vector. When it is determined thatthe object around the vehicle is a driver registered in the vehicle, thecontroller 40 may perform control such that doors of the vehicle arelocked/unlocked. Meanwhile, when it is determined that the object aroundthe vehicle is not the driver registered in the vehicle, the controller40 may perform control such that the doors of the vehicle are notlocked/unlocked.

The controller 40 may generate multi-modal data by matching the firstfeature vector and the second feature vector when determining that theobject around the vehicle is the driver, by comparing the object aroundthe vehicle and information about the driver registered in the vehicle,based on the extracted second feature vector.

More specifically, the controller 40 may match the extracted secondfeature vector of the driver (that is, the driver's facial features orcomplexion) and the body shape of the object around the vehicle and maygenerate the multi-modal data based on the matching first featurevector. Since the first feature vector is extracted from motions of allobjects around the vehicle as well as the driver, all first featurevectors extracted are not used to generate the multi-modal data, andonly the feature of the first feature vector that matches the driver'sfacial features or complexion extracted from the second feature vectormay be selected and used to generate the multi-modal data.

The controller 40 may determine the driver's emotional state, based onthe generated multi-modal data. For example, the controller 40 maydetermine the driver's current emotional state to be a pleasant state, abored state, or a sorrowful state.

When the driver's emotional state is determined, the controller 40 maygenerate a graphical user interface according to the emotional state andmay output the graphical user interface on the window display 30. Aspecific description thereabout will be given with reference to FIGS. 7to 9.

Meanwhile, when determining that the object around the vehicleapproaches the vehicle, but the face is not detected, the controller 40may provide, through the window display 30, feedback that the face isnot detected.

For example, when determining that the object around the vehicle is notlocated within the angle of view of the camera 20, the controller 40 maygenerate a graphical user interface for leading the object to be locatedwithin the angle of view of the camera 20. The controller 40 may resizea graphic according to the distance between the vehicle and the objecttherearound. For example, the controller 40 may make the output graphiclarger in size when the object around the vehicle is located fartheraway from the vehicle and may make the output graphic smaller in sizewhen the object around the vehicle is located closer to the vehicle.

When determining that the object around the vehicle is difficult todetect due to foreign matter on the camera 20 although being locatedwithin the angle of view of the camera 20, the controller 40 may providefeedback that the foreign matter exists on the camera 20, therebyenabling the face to be easily detected.

FIG. 2 is a flowchart illustrating a method for controlling a vehicleaccording to an embodiment of the present disclosure.

As illustrated in FIG. 2, in the vehicle control method according to theembodiment of the present disclosure, the controller 40 obtains a motionimage of an object around the vehicle and determines approach of theobject around the vehicle (Step S21). When the approach of the objectaround the vehicle is determined through the sensor 10, the controller40 determines whether a face is detected, based on the motion image(Step S22).

When it is determined in step S22 that the face is not detected, thecontroller 40 generates a graphical user interface to provide feedbackthat the face is not detected and outputs the graphical user interfacethrough the window display 30 (Step S23).

When determining that the object around the vehicle is not locatedwithin the angle of view of the camera 20, the controller 40 may, instep S23, generate a graphical user interface for leading the object tobe located within the angle of view of the camera 20. For example, thecontroller 40 may resize a graphic according to the distance between thevehicle and the object therearound. The controller 40 may make theoutput graphic larger in size when the object around the vehicle islocated farther away from the vehicle and may make the output graphicsmaller in size when the object is located closer to the vehicle.

When determining that the object around the vehicle is difficult todetect due to foreign matter on the camera 20 although being locatedwithin the angle of view of the camera 20, the controller 40 may, instep S23, provide feedback that the foreign matter exists on the camera20, thereby enabling the face to be easily detected.

The controller 40 obtains a facial image of the object around thevehicle from the camera 20 (Step S24).

The controller 40 extracts a second feature vector, based on the facialimage of the object obtained in step S24 (Step S25). Here, the secondfeature vector is intended to detect a facial expression. According toan embodiment, the second feature vector may include facial features(eyes, nose, and mouth) and a complexion.

The controller 40 determines whether the object around the vehicle is adriver registered in the vehicle, based on the second feature vectorextracted in step S25 (Step S26). When it is determined in step S26 thatthe object around the vehicle is the driver (Y), the controller 40performs control to lock or unlock doors of the vehicle (Step S27).Meanwhile, when it is determined in step S26 that the object around thevehicle is not the driver registered in the vehicle, the controller 40ends the process.

FIG. 3 is a flowchart illustrating a method for controlling a vehicleaccording to another embodiment of the present disclosure.

As illustrated in FIG. 3, in the vehicle control method according to theother embodiment of the present disclosure, the controller 40 determinesmotion of an object around the vehicle that is detected by the sensor 10(Step S31). The motion of the object around the vehicle in step S31 mayrefer to motions of all objects that move around the vehicle, and theobject may include a user.

When determining that there is the motion of the object around thevehicle, the controller 40 obtains a motion image of the object aroundthe vehicle from the camera 20 (Step S32).

The controller 40 extracts a first feature vector, based on the motionimage of the object around the vehicle (Step S33). Here, the firstfeature vector is intended to detect the feature of the motion of theobject around the vehicle. According to an embodiment, the first featurevector may include a specific gesture or a body shape of the objectaround the vehicle. A face may be included in the body shape. Since thedetected face does not have accuracy sufficient to determine whether theobject is a driver or not, it is not easy to determine whether theobject is the driver or not, based on only the body shape extracted asthe first feature vector.

The controller 40 determines approach of the object detected by thesensor 10 to the vehicle (Step S34). When determining that the objectaround the vehicle approaches the vehicle, the controller 40 determineswhether a face is detected, based on the motion image (Step S35).

When it is determined in step S35 that the face is not detected (N), thecontroller 40 generates a graphical user interface to provide feedbackthat the face is not detected and outputs the graphical user interfacethrough the window display 30 (Step S36).

When determining that the object around the vehicle is not locatedwithin the angle of view of the camera 20, the controller 40 may, instep S36, generate a graphical user interface for leading the object tobe located within the angle of view of the camera 20. For example, thecontroller 40 may resize a graphic according to the distance from theobject around the vehicle. The controller 40 may make the output graphiclarger in size when the object around the vehicle is located fartheraway from the vehicle and may make the output graphic smaller in sizewhen the object around the vehicle is located closer to the vehicle.

When determining that the object around the vehicle is difficult todetect due to foreign matter on the camera 20 although being locatedwithin the angle of view of the camera 20, the controller 40 may, instep S36, provide feedback that the foreign matter exists on the camera20, thereby enabling the face to be easily detected.

Meanwhile, when it is determined in step S35 that the face is detected(Y), the controller 40 obtains a facial image of the object around thevehicle from the camera 20 (Step S37).

The controller 40 extracts a second feature vector, based on theobtained facial image (Step S38). Here, the second feature vector isintended to detect a facial expression. According to an embodiment, thesecond feature vector may include facial features (eyes, nose, andmouth) and a complexion.

The controller 40 determines whether the object around the vehicle is adriver registered in the vehicle, based on the second feature vectorextracted in step S38 (Step S39).

When determining that the object around the vehicle is the driver, thecontroller 40 generates multi-modal data by matching the first featurevector and the second feature vector (Step S40).

In step S40, the controller 40 may match the extracted second featurevector of the driver (that is, the driver's facial features orcomplexion) and the body shape of the object around the vehicle and maygenerate the multi-modal data based on the matching first featurevector. Since the first feature vector is extracted motions of allobjects around the vehicle as well as the driver, all first featurevectors extracted are not used to generate the multi-modal data, andonly the feature of the first feature vector that matches the driver'sfacial features or complexion extracted from the second feature vectormay be selected and used to generate the multi-modal data.

The controller 40 determines the driver's emotional state, based on thegenerated multi-modal data (Step S41). In step S41, the controller 40may determine the driver's current emotional state to be a pleasantstate, a bored state, or a sorrowful state, based on the multi-modaldata.

When the driver's emotional state is determined, the controller 40generates a graphical user interface according to the emotional stateand outputs the graphical user interface on the window display 30 (StepS42).

FIG. 4 is a view illustrating the camera included in the vehicle controlapparatus according to an embodiment of the present disclosure.

As illustrated in FIG. 4, the camera 20 according to the embodiment ofthe present disclosure may be inserted into a center pillar (a B pillar)41 on a lateral side of the vehicle and therefore may easily obtain amotion image of an object around the vehicle and a facial image of theobject that closely approaches the vehicle.

FIG. 5 is a view illustrating the angle of view of the camera accordingto an embodiment of the present disclosure.

As illustrated in FIG. 5, the angle of view of the camera 20 accordingto the embodiment of the present disclosure ranges from 42° to 70°.Accordingly, the camera 20 may easily take facial and motion images ofan object that is located 30 cm from the vehicle.

FIG. 6 is a view illustrating a display type according to an embodimentof the present disclosure.

As illustrated in FIG. 6, the window display 30 of the presentdisclosure may include window displays 61 onto which images areprojected from projectors 62. Here, the projectors 62 are intended toproject images toward the window displays 61 on opposite sides of thevehicle. According to an embodiment, the projectors 62 may be includedin sun visors 63 inside the vehicle.

FIGS. 7 to 13 are views illustrating graphical user interfaces outputaccording to embodiments of the present disclosure.

FIGS. 7 to 9 are views illustrating graphical user interfaces that aredifferently generated according to a driver's emotional statesdetermined by the controller 40.

As illustrated in FIG. 7, according to an embodiment of the presentdisclosure, when determining the driver's emotional state to be apleasant state, the controller 40 may output an emoticon 72 having apleasant expression on a window display 71.

As illustrated in FIG. 8, according to an embodiment of the presentdisclosure, when determining the driver's emotional state to be asorrowful state, the controller 40 may output an emoticon 82 having asorrowful expression on a window display 81.

As illustrated in FIG. 9, according to an embodiment of the presentdisclosure, when determining the driver's emotional state to be a boredstate, the controller 40 may output an emoticon 92 having a boredexpression on a window display 91.

In addition, as illustrated in FIGS. 10 to 13, when determining that anobject around the vehicle approaches the vehicle, but a face is notdetected, the controller 40 may provide, through the window display 30,feedback that the face is not detected.

As illustrated in FIGS. 10 and 11, according to embodiments of thepresent disclosure, when the controller 40 determines that the objectaround the vehicle approaches the vehicle, but the face is not detectedbecause the object is not located within the angle of view of the camera20, the controller 40 may output, on displays 101 and 111, images forleading the face to be moved to the position of the camera 20 insertedinto the center pillar. As illustrated in FIGS. 10 and 11, the imagesfor leading the face to be moved may include arrows 102 and 112. Withoutbeing limited thereto, however, the images may be implemented in variousforms. Furthermore, the images may be output to have different sizesaccording to the distance. In addition, the images may include numbersthat lead the face to be moved and represent the distance between thevehicle and the object therearound.

For example, the controller 40 may increase the size of the arrow 102 asillustrated in FIG. 10 when determining that the object around thevehicle is far away from the vehicle and may decrease the size of thearrow 112 as illustrated in FIG. 11 when determining that the objectaround the vehicle is close to the object.

As illustrated in FIGS. 12 and 13, according to embodiments of thepresent disclosure, when an object around the vehicle is difficult todetect due to foreign matter on the camera 20 although being locatedwithin the angle of view of the camera 20, the controller 40 may providefeedback that the foreign matter exists on the camera 20, therebyenabling the face to be easily detected. For example, the controller 40may perform control to generate and output a text message “error” 122 ona window display 121 as illustrated in FIG. 12, or to generate andoutput a text message “wipe the lens” 132 on a window display 131 asillustrated in FIG. 13. However, a thing that is output for the feedbackis not limited to the text messages and may include images reflectingthe feedback contents. For example, the controller 40 may output animage on the window display 131 near the B pillar to provide feedbackthat the face is not detected due to the foreign matter, or may output ared circular image to provide feedback that the foreign matter exists onthe camera 20.

FIG. 14 is a block diagram illustrating a configuration of a computingsystem 1000 that executes a method according to an embodiment of thepresent disclosure.

Referring to FIG. 14, the computing system 1000 may include at least oneprocessor 1100, a memory 1300, a user interface input device 1400, auser interface output device 1500, a storage 1600, and a networkinterface 1700 that are connected together through a bus 1200.

The processor 1100 may be a central processing unit (CPU) or asemiconductor device that processes instructions stored in the memory1300 and/or the storage 1600. The memory 1300 and the storage 1600 mayinclude various types of volatile or non-volatile storage mediums. Forexample, the memory 1300 may include a read only memory (ROM) 1310 and arandom access memory (RAM) 1320.

Accordingly, the steps of the methods or algorithms described above inrelation to the embodiments disclosed herein may be directly implementedwith a hardware module or a software module executed by the processor1100, or a combination thereof. The software module may reside in astorage medium (that is, the memory 1300 and/or the storage 1600) suchas a RAM memory, a flash memory, a ROM memory, an EPROM memory, anEEPROM memory, a register, a hard disk, a detachable disk, or a CD-ROM.The exemplary storage medium may be coupled to the processor 1100, andthe processor 1100 may read information out of the storage medium andmay record information in the storage medium. Alternatively, the storagemedium may be integrated with the processor 1100. The processor 1100 andthe storage medium may reside in an application specific integratedcircuit (ASIC). The ASIC may reside in a user terminal. In another case,the processor 1100 and the storage medium may reside in the userterminal as separate components.

According to the embodiments of the present disclosure, the vehiclecontrol apparatus and method may output a graphical user interface toprovide feedback that a face is not detected or may output a graphicaluser interface differently generated according to an emotional state,thereby providing an aesthetic value to a user and improving themerchantability of a vehicle.

Hereinabove, although the present disclosure has been described withreference to exemplary embodiments and the accompanying drawings, thepresent disclosure is not limited thereto, but may be variously modifiedand altered by those skilled in the art to which the present disclosurepertains without departing from the spirit and scope of the presentdisclosure claimed in the following claims.

Therefore, the exemplary embodiments of the present disclosure areprovided to explain the spirit and scope of the present disclosure, butnot to limit them, so that the spirit and scope of the presentdisclosure is not limited by the embodiments. The scope of the presentdisclosure should be construed on the basis of the accompanying claims,and all the technical ideas within the scope equivalent to the claimsshould be included in the scope of the present disclosure.

What is claimed is:
 1. An apparatus for controlling a vehicle, theapparatus comprising: a sensor configured to detect approach of anobject around the vehicle; a camera configured to obtain a motion imageand a facial image of the object around the vehicle; and a controllerconfigured to generate a graphical user interface (GUI) based on theobtained motion and facial images of the object around the vehicle. 2.The apparatus of claim 1, wherein the controller is configured toextract a first feature vector based on the motion image of the objectaround the vehicle.
 3. The apparatus of claim 2, wherein the firstfeature vector includes a specific gesture or a body shape of the objectaround the vehicle.
 4. The apparatus of claim 2, wherein the controlleris configured to extract a second feature vector from the facial imageand to determine whether the object is a driver based on the secondfeature vector.
 5. The apparatus of claim 4, wherein the second featurevector includes a facial expression or a complexion.
 6. The apparatus ofclaim 4, wherein the controller is configured to generate multi-modaldata, based on the second feature vector and the first feature vectormatching the second feature vector.
 7. The apparatus of claim 6, whereinthe controller is configured to determine an emotional state of thedriver based on the multi-modal data.
 8. The apparatus of claim 7,wherein the controller is configured to generate the graphical userinterface (GUI) according to the emotional state.
 9. The apparatus ofclaim 4, wherein the controller is configured to controllocking/unlocking of doors of the vehicle when determining that theobject around the vehicle is the driver.
 10. The apparatus of claim 1,wherein the controller is configured to determine whether a face of theobject around the vehicle is detected, based on the motion image, and togenerate a graphical user interface to provide feedback on a reason whythe face is not detected when determining that the face is not detected.11. The apparatus of claim 1, further comprising a window display onwhich the graphical user interface (GUI) is output.
 12. A method forcontrolling a vehicle, the method comprising: obtaining a motion imageand a facial image of an object around the vehicle; determining adriver's emotional state, based on the obtained motion and facial imagesof the object around the vehicle; and outputting a graphical userinterface generated based on the driver's emotional state.
 13. Themethod of claim 12, further comprising extracting a first feature vectorfrom the motion image of the object around the vehicle after obtainingthe motion image of the object around the vehicle.
 14. The method ofclaim 13, wherein the first feature vector includes a specific gestureor a body shape of the object around the vehicle.
 15. The method ofclaim 13, further comprising extracting a second feature vector from thefacial image of the object around the vehicle and determining that theobject is the driver based on the second feature vector, the extractingperformed after obtaining the facial image of the object around thevehicle.
 16. The method of claim 15, wherein the second feature vectorincludes a facial expression or a complexion.
 17. The method of claim15, wherein determining the driver's emotional state based on theobtained motion and facial images of the object around the vehiclecomprises: generating multi-modal data based on the second featurevector and the first feature vector matching the second feature vector;and determining the driver's emotional state based on the multi-modaldata.
 18. The method of claim 15, further comprising controllinglocking/unlocking of doors of the vehicle when determining that theobject around the vehicle is the driver.
 19. The method of claim 12,wherein the graphical user interface is output differently according tothe driver's emotional state when outputting the graphical userinterface generated based on the driver's emotional state.
 20. A methodfor controlling a vehicle, the method comprising: obtaining a motionimage of an object around the vehicle; determining whether a face of theobject around the vehicle is detected based on the motion image; when aface of the object is detected: obtaining a facial image of the objectaround the vehicle; determining a driver's emotional state, based on theobtained motion and facial images of the object around the vehicle; andoutputting a graphical user interface generated based on the driver'semotional state; and when a face of the object is not detected,outputting a graphical user interface to provide feedback on a reasonwhy the face is not detected.